SUSSILLO, David  Computation thru Dynamics March 19 2013

PhD in 2009 at CMU...

today's title: computation thru dynamics
 using recurrent neural networks to unveil mechanisms in neural circuits

he works with Valerio Mante and Bill Newsome and in the Shenoy Lab...

did a post-doc in  Wolfgang Maass Lab..

sl: 1 (slide 1)  table of contents

intro
then training recurrent neural networks:   RNNs
then understanding how RNNs work, then 
contextual decision making  and
future directions...

work with Justin  Foster  in Krishna Shenoy Lab:

consider a monkey  freely behaving...
Foster  IEEE EMBS  2012

he shows a complex parallel recording of spike data from this freely behaving macaque.

so, what do you do with this data?

a suggestion from one of his profs... (in jest)
throw away all the channels except one...(!)

looking at firing rates of many neurons over time ( in ms)

perhaps extract  a few principal components... over time..

"I work at the level of  firing rates because we can make networks do interesting computations"

(possibly a quote from Larry Abbott:  what are the biophysical correlates of these firing rates...
 he mentions it but will not deal with it  (the biophys correlates and spikes...
all  David's work  uses  firing rates)

RNNs  = recurrrent neural wroks   

spikes per sec  as a function of time

crucially, a saturating nonlinearity...

tau *  Xprime  = -X  +  Jr  + By

nonlinear, distributed...

r  =  [tanh(x)]  
z =  Wr

employs feedback

see sompolinsky  PRL in 1988 and
Rajan...

Sussillo and Abbott...

training  RNNs is really hard...

eg consider a toy example:

..train it to emit a triangular sinusoid  (sawtooth pattern):

how can it possibly do that?

Training  RNNs  use supervised learning...

defined by  error = target - output

modify weights  to minimize errors

FORCE learning  (real world perspective)

sussillo and abbott, Neuron 2009  and also PLOS One in 2012...

problem is this is really slow learning  ... forget it... too slow...
(ie  simple difference learning (as in backprop) is too slow)

clearly learning doesn't work that way in real life

initialize randomly...

but note: as soon as learning turns on then the

weights change to compensate for the inadequacy of the network...

z(t)  is the output  (the triangular sine wave eg)

he shows a network he was modeling:

this was a model of a 96 dof running skeleton with 96 joint angles.

clip shows a running skeleton...

change inputs and system can change to walking...

with statistical contextual input...

major criticism of these kinds of ML models:  what the heck is going on???
(the CS/ math guys don't care as long as it works)

in collab with  Omri Barak... he also did the following

toy example..

how does a sine-wave generator work...

he states that the output freq  can increase...

look at the PCs  = principle comps...

see oscillations among the PCs.

where are the fixed points?  and the attractors.

why is this important?

Because you can use fixed points and attractor basins to encode memory...  

but it's also impt...  cuz u can linearize around  a fixed point...

can have decay to the fixed point...

can have repellers:  (that repel away from the dynamics)

and saddle points...

Omri and David  use this type of language...

find a fixed point in the middle  of an oscillating attractor...

a context dependent fixed point ? allows you to alter context.

so,  here are his conclusions from tech part of this talk:

RNNS are a natl model class for modeling cortical phenom:  dynamical, nonlinear, distributed

can use a bunch of  RNNS  to tile the state space...

with Valerio Mante  and Bill Newsome...

both thought abt this in experimental (expt) context...

contextual decision making...

computations in cortical circuits are flexible...

can use context...  (eg  prior  expectations, goals,  etc)

PFC  contributes to flexibility of   decisions...(decns)

allows you to attend to relevant  stim...

He shows a random dots task  variant:

context-dependent gating in  monkeys...

a contextual motion task  (a la Bill Newsome  coherent motion:  strong L, strong R or random eg)

but also presents dots that are in color  100% green or 100% red or everything in btwn...

task:  designate direction of most green or most red dots...

(this task is similar to Newsome task ,but dots are also colored, so sometimes
monkey must attend to direction of motion and sometime to direction of red dots or green dots)

he shows a graph:  

Y axis:  color coherence  eg  0.6  to -0.6

and  X axis  direction coherence  (a big range)

monkey CAN do this task...

sigmoid curve for motion to the right... (as a func. of coherence)

color trials...  exactly the same but ignoring  movement
 and only using color:  sigmoid curve  for color (when ignoring motion)

area MT encodes info abt direction of stim

V4,  IT  encode info about COLOR of stim..

decision whether to saccade to the L or R is encoded in   LIP  PFC, SC (super collic);  integrating evidence...

Valerio initially looked in MT: 
 to our surprise:  he did NOT see differentiation between
color and  direction trials  (no diff in firing rates)

so, it appears that relevant stim is selected LATE in the game in PFC...
(high upstream in the processing sequence)

note the richness of dynamics in FEF = frontal eye field

rich temporal variety   (the prefrontal zoo)  (he is refering to data in FEF)

mixed signals in FEF neurons...

Y axis  = motion  R**2
X axis = choice R**2

all over  the place  (big fat scattergram cloud)

now de-mix the responses at the level of the population

can u break down what u see into 3 separate patterns (eg)
by doing  dimensional  reduction...

look at weights applied to each pattern  over time

pattern 1 = choice (monkey actually made)  vs  motion of dots  vs  color of dots.

look at motion  trials only

he examines a 750 ms: window :  dots on to dots off

and look at FEF responses  in the monkey...

next look at the motion evidence to the R or L...

DO see strong  signal that indicates
INTEGRATION of  motion evidence in a motion trial
(looking only at correct trials)

he rotates his diagram by 90 degr in one acis  and you see similar  curves...

see big evidence of  color representation on the color  trials...

(but irrelevant stim is ALSO repres  (to our surprise))

irrelevant  dimension IS repres in this population despite the fact that it is irelevant...

context is a 4th dimension

the first 3 Ds  are  motion, color, and choice...

according to an attention model:
during the motion task, color should not even make it into higher centers... and v v s.

so how does selective integration occur?

conclusions from the data:

task-relevant vars are mixed in the response of single neurons

irrelevant inputs are not filtered out

but rather, selection occurs LATE in the PFC!

now... how COULD selective integration occur?

a neural-network model of selective integraiton

choice is either  +1 or -1  to  indicate choice...

hessian-free learning...

network needs to do what the brain is doing...

trained network creates a bounded integrator...

network output (in red)  tracks a bounded integrator and when it hits its asymptote it stays put...

now examine the color trials... look at the  model's  trajectories (in state space)

look at strong evidence either toward the R or L.

strong evidence toward the R eg  does incr the choice toward the R...

drift diffusion model... works exactly the same way..

they just model 750 ms of data...

when you rotate 90 degr  ('bcuz color is irrelevant during motion trials)  the color data is still there.

LINE ATTRACTOR...   one eigenvalue..  that is linear...

eg see Seung  PNAS  1996  to understand oculomotor system..  (using a line attractor)

fixed points make a line attractor  (if u kick sys. it comes right back)

can use  two line attractors for two difft contexts...

the line attractors are context dep.  (eg  color is relevant and motion ignored and v v s)

look at a pulse of color info  and it descends  the linear attractor

it is NOT just  a projection onto the line attractor...

we know there is a choice axis that is a line attractor
 and both color and motion project onto it...

but this is a NON NORMAL  system:  R and L eigenvectors are not the same...

selection vector is an addit.  dof = degr of freedom...

all that is changing is one vector  

take home model picture:  this is like the "receptive field of our task"

selection vector for motion is aligned with motion and v v s...

a prediction of this model...

if u could give pulse of color info... would see  relaxation (? toward that attractor)...

conclusions from model

the model represents both relevant and irrel inputs in separate dimensions

2 contextually dependent line attractos are responsible for the integration...

gating of sensory signals
does not require modulation of sensory responses

it is not abt  suppressing the irrel input but
 abt selecting the relevant input in state space

mixes, separable reps are contextually and dynamic linked to generate. the final output...

*** future work:

(usual line of research is ...)
modeling framework

say, have a task...  and have rodent or monkey...= system

get the neurobio data (firing rates from a pop. of neurons)

guess mechanism then make a model  and
 have the model generate simulated data and see if there is a match...

but what should the solutions look like...

eg consider a generic router  with input  vector and output vector...
 eg the pulvinar...or...

eg to do modeling motor cortex during reaching  

sussillo, churchland, kaufman and shenoy  in prep

and churchland and cunningham in Nature 2012

look at monkey reach: while recording emg  from deltoid and pect...

dynamic systems hypothesis

do  simM1  and  simSC1-3  (sims of M1 and of  Super Collic)

they found rotational dynamics in M1...

but what we found...
v similar story...  when network goes

it generates  a single fixed point which it oscillates around...

look for rotational dynamics...

he uses standard least squares
eg  Marens and Sutskever  ICML  2011

fit the EMG with as few wiggles as possible

the actual solution is a confederacy of crazy wiggles...

but  when simplified...  there are a large # of oscillations
 that are responsible for generating the EMG...

finally he shows  a  BMI interface
(monkey following a dot with its mind)

employs a Utah array in M1/PMd

last slide:  he wants to add biophys to model..
wants to introduce dynamics into the synapses

what is  J(ik)  a random synapse  as an arbit number)
...  to the math guys  its just an arbitrary number  (synapses are just weights)

BUT  synapses are different  (depends on histology, detailed microanat)

but brain brings together difft alliances/ collabs at difft times...

what info is propagated?

he shows collabs:
Krishna Shenoy:  Mark Churchland and Matt Kaufman

Bill Newsome Lab:  Valerio Mante  et al...

Q:  how  does optimization work?
A  suppose  optimiz  does not work out... a conflict... say, the expt data/monkey spaces out...
then my model does not speak to it...
my model gives a v idealized  version of what the monkey is doing...
can easily imagine a situation in  which selection vector is not that good..

***

Q  what's the diff btwn guessing mechanism  vs  optimized model and discovering mech...
A: prob not a unique solution for anything...
I need to maximally simplify my models and then get a unique soln...

Q (Jay):  now that you've presented a low d model, how abt
developing a model/ explanation only based on those dimensions?
then try to explain anomalies...
A  my model may have missed the anomalies..
I know the fixed points ; I know the dynamics; and the delay modes
I know how it's working...
should be able to write down a simpe set of equations...

Q  can you score models in some bio relevant way?  eg. add cost functions  or  
include metric related to the model  eg complexity... (a la Kolmogorov)
A  I showed a regularizaiton based on... 
could try to match low  firing rates...
in ML.. what are the sparse  regularizaitons doing... enabling system to jump to a low d maniforld...

Q (Kwabena Boahen):  confidence in your models?
A  I;'m pretty confident  of what I've got
in one part of the state space the linear dynamcis are so (holds up fingers) and in another part of the ss
they are  so... eg  orthogonal...

someone comments about  use of language in which S learns to do the task
in one trial or just with directions.    (not something he modeled)

Q (Bill Softky)  beautiful  demo  and nice demo of attractors.
but in ur task: u have rinsed a lot of complexity out of system...
(cuz ur giving monkey forced choice task)...
his Q : is this really what the brain does?
A:  I'm only making a claim abt one tiny circuit  (no claim of generality)

Q  (Surya G)  is there a data set that u could NOT explain?
A  u prob could make give me a data set that I cud not explain but
these are v powerful models...

Q/comment (jay): I'm thinking about the literature pertaining to human attention
:  the SHADOWING TASK: 
different messages simultaneously delivered to each ear...
S must repeat back what he hears  in the L ear and not the R ear  etc...
big debates in literature  (abt bleed over  from  unattended ear);
so perhaps  we don't just settle to a single fixed point,
trying to understand...  how this would apply in a world that's always changing...
A:  in our expt  the data is always changing...
but does not mean that the underlying structure that implements the  transform from
noisy evidence to integrated choice changes.
(a way of thinking abt  this)
fixed point in the middle that is surrounded by an oscillating dynamic...

(end at  4:24 PM:  Clark Auditorium:)